Consumer Reports does another strange photo print test

For 80 years, Consumer Reports has been the bellwether of reliable consumer product reviews. From washers and dryers to cars and trucks, CR was the go-to resource for unbiased information. Every year, the annual buyers guide – available on newsstands – influenced camera purchases, as the reviewers were seen as unbiased and truthful. Sometimes the recommendations were head-scratchers – like year the publication recommended a discontinued Yashica camera, sending dealers scrambling to find it for customers who “had to have” the camera earning the CR recommendation.

In the digital age, with review sites everywhere and the decline in influence of annual buying guides on consumer perception, Consumer Reports still follows the same editorial and business model. It’s laudable, really, but in the case of a recent “review” of “the best photo printing services,” it went from “laudable” to “laughable.”

The review limited the scope of the “test” to online photo printing sites, and the criteria was decidedly unscientific:

“To simulate the average consumer’s experience, we asked CR staffers to contribute their digital photos. We ordered prints in various sizes and finishes, then tasked our panel of trained photo experts with evaluating the results using the protocol we employ in our printer labs.

“Our testers rated each print on 20 metrics, including color saturation, sharpness, the accuracy of the flesh tones, lack of streaking or graininess, and the breadth of the image’s blacks and whites. All that data was then telescoped into a numerical score for color, clarity, and contrast.”

Left unsaid were the 20 metrics, which appear to be purely subjective. Given the results, it would be helpful to know what they were, especially since in many cases, the brands are using the same wholesale print lab.

The result?

Walgreens                  98
AdoramaPix                97
Walmart Photo           91
Amazon                      88
CVS Photo                  88
Shutterfly                    81
Snapfish                      81
Costco Photo Center  72
Nations Photo Lab     68
Mpix                           54

Especially strange was the ranking of MPix and Nations Photo Lab as the worst of the roundup. A slight spread in the ratings would be explainable, but it’s hard to believe the quality of these pro labs is half that of Walgreens. That just defies reasonable expectations. And this shows just how damaging these uniformed reviews can be. I think it hurts the reputation of Consumer Reports more than the photo labs.